Scaling Up Influence Functions
نویسندگان
چکیده
We address efficient calculation of influence functions for tracking predictions back to the training data. propose and analyze a new approach speeding up inverse Hessian based on Arnoldi iteration. With this improvement, we achieve, best our knowledge, first successful implementation that scales full-size (language vision) Transformer models with several hundreds millions parameters. evaluate in image classification sequence-to-sequence tasks tens hundred examples. Our code is available at https://github.com/google-research/jax-influence.
منابع مشابه
Rod influence on hue-scaling functions
Rod influence on hue appearance of spectral lights was characterized by comparing the scaling of red, green, yellow, and blue hue sensations for an 8 degrees-diameter, 7 degrees-eccentric test spot under conditions that minimized (cone plateau) and maximized (dark adapted) rod influence at two mesopic light levels (1.5 and 3.0 log scoptic trolands). At the lower light level, the hue-scaling fun...
متن کاملThe up-scaling of ecosystem functions in a heterogeneous world
Earth is in the midst of a biodiversity crisis that is impacting the functioning of ecosystems and the delivery of valued goods and services. However, the implications of large scale species losses are often inferred from small scale ecosystem functioning experiments with little knowledge of how the dominant drivers of functioning shift across scales. Here, by integrating observational and mani...
متن کاملExploiting submodular value functions for scaling up active perception
In active perception tasks, an agent aims to select sensory actions that reduce its uncertainty about one or more hidden variables. For example, a mobile robot takes sensory actions to efficiently navigate in a new environment. While partially observable Markov decision processes (POMDPs) provide a natural model for such problems, reward functions that directly penalize uncertainty in the agent...
متن کاملScaling Up
Most analytical and experimental results on evolutionary programming (EP) are obtained using low-dimensional problems, e.g., smaller than 50. It is unclear, however, whether the empirical results obtained from the low-dimensional problems still hold for high-dimensional cases. This paper investigates the behaviour of four diierent EP algorithms for large-scale problems, i.e., problems whose dim...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the ... AAAI Conference on Artificial Intelligence
سال: 2022
ISSN: ['2159-5399', '2374-3468']
DOI: https://doi.org/10.1609/aaai.v36i8.20791